Electronic instruments, including hardware-based and software-based musical instruments, offer an unlimited palette of expressive sounds. However, a performer's ability to access and control these sounds is limited by the music performance controllers through which the performer interfaces with the electronic instruments.
Many music performance controllers provide piano-style keys as note triggers. While well-suited for triggering percussive and plucked sounds, such controllers provide limited or no facility for controlling the expression of sounds once they are triggered. For example, common to all acoustic wind and bowed-string instruments is the performer's ability to continuously control each note's volume, pitch (e.g., as in a vibrato or a slide from one note to another), and timbre (bite pressure on the reed, or bow angle or bow position). It is often desired to have similar control of volume, pitch, and timbre when emulating wind and bowed-string instruments as well as when generating other emulative and non-emulative (e.g., synthetic) sounds.
Many piano-type controllers include additional controls, e.g., pitch-bend wheels, modulation wheels, foot pedals, breath controllers and pressure and/or aftertouch, to control the expression of a sound. With the exception of pressure/aftertouch, these ancillary controllers can be awkward to use as they are separated from the finger that determines which note to play. In other words, to modulate the volume, pitch, and timbre of a sound requires at least one hand, foot or mouth in addition to the hand used to select a note to be played. For example, one could control volume with pressure (channel or key), pitch with a pitch-bend wheel, and timbre with a foot pedal. However, it can be very awkward to control the expression of sound using two or three separate controllers.
Another problem with requiring more than one hand to play a note expressively is that the techniques that work (more or less well) for one note do not scale to two or more notes. For example, when three notes are sounding, it is not usually possible to modulate the volume, pitch or timbre of one of the notes without affecting the others. For another example, one cannot normally use a pitch wheel to slide from an A-minor (ACE) chord to a C-major (CEG) chord since the A-C slide is three semitones and the C-E slide is four semitones. One cannot normally apply a vibrato to a melody note without applying it to a concurrently sounding accompaniment chord.
In order to control expression polyphonically, that is, independently control the expression of more than one note at a time, the finger that plays the note can also control expression. For polyphonic multi-dimensional note expression, the finger that plays the note should control multiple (two or more) axes of expression for that note.
One approach to polyphonic multi-dimensional note expression is to provide a multi-dimensional sensor for each of plural note triggers. However, the large number of multi-dimensional sensors required for such a controller to have a wide (several octaves) note range can be costly. Furthermore, controllers that use a separate sensor for each note are not well-suited to controlling continuous slides from one note to another.
Another approach to polyphonic multi-dimensional note expression is to use a continuous multi-touch sensor to detect and track finger position. In order to provide the precision required for musical expression, many positions on the multi-touch sensor would have to be resolved, e.g., by scanning them sequentially. This scanning would have to be repeated frequently enough to track motion. Also, a touch would cover many positions that would have to be resolved to provide sufficiently fine tracking for musical expression purposes; accordingly, some sort of centroid determination or other mathematical processing would be required to identify a single position for each touch at any given time.
While such processing is readily performed on smartphone and tablet touchscreens, the cost can become excessive when scaled to surfaces sufficiently large for two-handed playing of a musical instrument. Furthermore, most touchscreens do not sense finger pressure, which is important for controlling a musical note's volume over time, and those few touchscreens that do sense pressure do so with insufficient pressure range for musical purposes.
Accordingly, what is needed is a cost-effective music performance controller with polyphonic multi-dimensional note expression.